Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new rule to TPV to workaround wrong remote resource selected or recorded in Galaxy DB #955

Merged
merged 7 commits into from
Oct 19, 2023

Conversation

sanjaysrikakulam
Copy link
Member

@sanjaysrikakulam sanjaysrikakulam commented Oct 19, 2023

I tested it using tpv dry-run with scenarios None, the name of an existing remote resource, and the name of a non-existing remote resource. It fails and shows a message when a non-existing remote resource is used.

Example:

    rules:
      - if: |
          # This rule will apply when users have non-exsisting remote resources selected or recorded in Galaxy DB
          retval = False
          remote_resource_tag = 'dummy-non-existing'
          remote_resource_destination = [d.dest_name for d in mapper.destinations.values() if any(d.tpv_dest_tags.filter(tag_value=remote_resource_tag))]
          if not remote_resource_destination:
              retval = True
          retval
        fail: |
          Invalid 'Remote resources id' selected in the config menu under 'User -> Preferences -> Manage Information'.
 Please reselect either 'default' or appropriate remote resource.

Fail output:

(venv) [galaxy@galaxyesginstance ~]$ tpv -v dry-run --job-conf /opt/galaxy/config/job_conf.yml
Traceback (most recent call last):
  File "/opt/galaxy/venv/bin/tpv", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/commands/shell.py", line 140, in main
    return args.func(args)
           ^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/commands/shell.py", line 57, in tpv_dry_run_config_files
    destination = dry_runner.run()
                  ^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/commands/dryrunner.py", line 20, in run
    return gateway.map_tool_to_destination(self.galaxy_app, self.job, self.tool, self.user,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/rules/gateway.py", line 51, in map_tool_to_destination
    return ACTIVE_DESTINATION_MAPPER.map_to_destination(app, tool, user, job, job_wrapper, resource_params,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/core/mapper.py", line 168, in map_to_destination
    evaluated_entity = self.match_combine_evaluate_entities(context, tool, user)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/core/mapper.py", line 138, in match_combine_evaluate_entities
    evaluated_entity = combined_entity.evaluate_rules(context)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/core/entities.py", line 489, in evaluate_rules
    rule = rule.evaluate(context)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/galaxy/venv/lib/python3.11/site-packages/tpv/core/entities.py", line 750, in evaluate
    raise JobMappingException(
galaxy.jobs.mapper.JobMappingException: Invalid 'Remote resources id' selected in the config menu under 'User -> Preferences -> Manage Information'. Please reselect either 'default' or appropriate remote resource

I have also tested the same by adding a dummy remote resource and ran 3 jobs each time updating my user preferences to default a test pulsar instance and dummy. It fails only when dummy is used. Feel free to test it via https://usegalaxy.esgwps.uno/ before merging (just to be sure, also for testing, you can use the fasta stats tool).

.. resource selected or recorded in Galaxy DB
@mira-miracoli
Copy link
Contributor

mira-miracoli commented Oct 19, 2023

It looks like a good solution to me, but when I wanted to test it on the ESG test Galaxy server, I got an error for the data upload:

Encountered an unhandled exception while caching job destination dynamic rule.

EDIT:
It worked when I logged in and selected a destination in the user preferences.
With the default settings for new users, it does not work

Copy link
Contributor

@kysrpex kysrpex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to test it via https://usegalaxy.esgwps.uno/ before merging (just to be sure, also for testing, you can use the fasta stats tool).

I trust your test. It's great that you tested it with None as well because that was a probable point of failure.

I requested changes because the rule would crash when anonymous users submit jobs (this is why the TPV dry-run fails).

files/galaxy/tpv/tool_defaults.yml Outdated Show resolved Hide resolved
files/galaxy/tpv/tool_defaults.yml Outdated Show resolved Hide resolved
files/galaxy/tpv/tool_defaults.yml Outdated Show resolved Hide resolved
sanjaysrikakulam and others added 4 commits October 19, 2023 10:18
.. attribute is not available and assume None as default
Co-authored-by: José Manuel Domínguez <[email protected]>
Co-authored-by: José Manuel Domínguez <[email protected]>
@sanjaysrikakulam
Copy link
Member Author

sanjaysrikakulam commented Oct 19, 2023

Addressed both @kysrpex and @mira-miracoli comments. Thanks for testing; check once again.

files/galaxy/tpv/tool_defaults.yml Outdated Show resolved Hide resolved
Co-authored-by: José Manuel Domínguez <[email protected]>
@sanjaysrikakulam sanjaysrikakulam merged commit 26574fe into usegalaxy-eu:master Oct 19, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants